{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "### CSQA数据集分析"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### CSQA数据集下载\n",
    "[CSQA训练集下载](https://s3.amazonaws.com/commensenseqa/train_rand_split.jsonl)\n",
    "\n",
    "[CSQA验证集下载](https://s3.amazonaws.com/commensenseqa/dev_rand_split.jsonl)\n",
    "\n",
    "[CSQA测试集下载](https://s3.amazonaws.com/commensenseqa/test_rand_split_no_answers.jsonl)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### CSQA训练集内容一览"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "outputs": [
    {
     "name": "stdout",
     "text": [
      "The sanctions against the school were a punishing blow, and they seemed to what the efforts the school had made to change?\n\nA:ignore\nB:enforce\nC:authoritarian\nD:yell at\nE:avoid\n\n\ntrue answer:  A\n-------------------------------------------------------------------------------------------------\nSammy wanted to go to where the people were.  Where might he go?\n\nA:race track\nB:populated areas\nC:the desert\nD:apartment\nE:roadblock\n\n\ntrue answer:  B\n-------------------------------------------------------------------------------------------------\nTo locate a choker not located in a jewelry box or boutique where would you go?\n\nA:jewelry store\nB:neck\nC:jewlery box\nD:jewelry box\nE:boutique\n\n\ntrue answer:  A\n-------------------------------------------------------------------------------------------------\nGoogle Maps and other highway and street GPS services have replaced what?\n\nA:united states\nB:mexico\nC:countryside\nD:atlas\nE:oceans\n\n\ntrue answer:  D\n-------------------------------------------------------------------------------------------------\nThe fox walked from the city into the forest, what was it looking for?\n\nA:pretty flowers.\nB:hen house\nC:natural habitat\nD:storybook\nE:dense forest\n\n\ntrue answer:  C\n-------------------------------------------------------------------------------------------------\n"
     ],
     "output_type": "stream"
    }
   ],
   "source": [
    "import jsonlines\n",
    "with open(r\"F:\\dataset_download\\CSQA\\train_rand_split.jsonl\", \"r+\", encoding=\"utf8\") as f:\n",
    "    cnt = 5\n",
    "    for item in jsonlines.Reader(f):\n",
    "        print(item['question']['stem'] + '\\n')\n",
    "        for i in range(5):\n",
    "            print(item['question']['choices'][i]['label'] + ':' + item['question']['choices'][i]['text'])\n",
    "        print('\\n')\n",
    "        print('true answer: ', item['answerKey'])\n",
    "        print('-' * 97)\n",
    "        cnt -= 1\n",
    "        if cnt == 0: break\n",
    "        \n",
    "\n"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n",
     "is_executing": false
    }
   }
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  },
  "pycharm": {
   "stem_cell": {
    "cell_type": "raw",
    "source": [],
    "metadata": {
     "collapsed": false
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}